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(54) IMAGE ALIGNING METHOD 

(57)Abstract: 

PROBLEM TO BE SOLVED: To enable the exact alignment of an 
image by adding mutual correlation coefficients in the longitudinal and 
lateral directions of a search area and defining a position, where these 
added values respectively become maximum values, as a matching 
position. 

SOLUTION: Concerning the search area in an input image divided into 
the blocks of M*N picture elements, inside that search area, mutual 
correlation coefficients C (a, b) at respective positions are operated 
according to a specified expression. Then, the operated results of the 
mutual correlation coefficients are stored in arrangement R (a, b) (S1). 
Next, the arrangement R (a, b) is added in (x) direction, SX(1) to SX 
(M-X+1) are added in (y) direction, and SY(1) to SY(N-Y+1) are 
respectively provided (S2). Then, among the SX(1) to SX(M-X+1) and 
among the SY (1) to SY(N-Y+1), the positions to respectively take the 

maximum values, namely, a column number xO and a row number yO are extracted (S3). Finally, the 
column number xO and the row number yO are decided as the matching position (S4). 
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CLAIMS 



[Claim(s)] 

[Claim 1] A predetermined criteria image is moved two-dimensional by the seek area in an input image. In each 
location of the above-mentioned seek area In the image alignment approach which makes the location where the cross 
correlation function of the above-mentioned criteria image and the above-mentioned input image is calculated, and this 
cross correlation function serves as the greatest value the matching position of the above-mentioned criteria image and 
the above-mentioned input image The image alignment approach characterized by making into the above-mentioned 
matching position the location where the above-mentioned cross correlation function is added about the lengthwise 
direction and longitudinal direction of the above-mentioned seek area, and the aggregate value of this lengthwise 
direction and the aggregate value of this longitudinal direction turn into the greatest value, respectively. 
[Claim 2] the above-mentioned aggregate value - the square of the above-mentioned cross correlation function ~ the 
image alignment approach according to claim 1 which is the aggregate value of a value. 
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DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention] This invention relates to the alignment approach of the image in the automation technique 
which used the image processing, for example, a character recognition technique, the automation technique of the 
visual inspection of a product, etc. 
[0002] 

[Description of the Prior Art] With the automation technique using an image processing, when it is the relation with 
which a predetermined criteria image (template) and a predetermined input image lap in a parallel displacement, the 
above-mentioned criteria image is moved within the seek area prepared into the input image, and the actuation which 
looks for the location which existed best, i.e., template matching,, is used abundantly. In the case of matching, it is 
important to ask for whether the above-mentioned criteria image matches in which location of the above-mentioned 
input image correctly. For this reason, in the automation technique using an image processing, the cross-correlation 
coefficient method which calculates the cross correlation function between the above-mentioned criteria image and the 
above-mentioned input image has been used well, moving the above-mentioned criteria image within the seek area of 
an input image. Drawing 3 is drawing explaining template matching which used the conventional cross-correlation 
coefficient method. A cross-correlation coefficient C (a, b) calculates moving the seek area in the input image which 
consists the criteria image of a XxY pixel of a MxN pixel, as shown in drawing 3 (a). If the coordinate at the upper left 
of a criteria image (a, b) moves in the range of a coordinate (1 1) to a coordinate (M-X +1, N-Y+l) and the above- 
mentioned cross correlation function calculates, cross-correlation-function MATORIKUSSU as shown in drawing 3 (b) 
will be obtained, for example, in drawing 3 (b), since C (xO, yO) is max in 0.9, a criteria image and the same image 
exist in the location of a=x0 and b=y0 -- ** (it matched) ~ it is judged. In addition, about such a Prior art, it is indicated 
by "image-processing handbook" p303-p304 of Morio Onoe Mr. edit, for example. 
[0003] 

[Problem(s) to be Solved by the Invention] However, the conventional cross-correlation coefficient method which was 
described above is sensitive to a noise, and since the maximum of the above-mentioned cross correlation function 
becomes indistinct, there are many possibilities of obtaining the mistaken result. In addition, a noise here is a noise in 
the semantics of a wide sense including the difference produced since the lighting conditions when picturizing the 
difference which always exists between the others and above-mentioned criteria images and the above-mentioned input 
images which are generated with an image incorporation camera or an image processing system, for example, the 
above-mentioned criteria image and the above-mentioned input image, changed. [ noise ] The example from which 
matching exact for the above-mentioned noise is not obtained is explained using drawing 4 . Drawing 4 is a graph for 
explaining the incorrect judging in the conventional cross-correlation coefficient method with which the above- 
mentioned noise becomes a cause here. For example, in MATORIKUSSU of the cross correlation function shown in 
drawing 4 , although a cross correlation function takes the greatest value in a location (x0-l, yO), this is because the 
above-mentioned noise which existed in a criteria image and input image information lapped in the location [ by 
chance / (x0-l, yO) ], and right matching positions are (xO, yO). It sets in such a situation, and in the conventional cross- 
correlation coefficient method, since the decision of a matching position is made only with the greatest above- 
mentioned value and surrounding correlation is not taken into consideration, it will be judged that (x0-l, yO) are 
matching positions. In order to solve the technical problem in a Prior art which was described above, this invention 
improves a cross-correlation coefficient method, and it aims at offering the approach of performing alignment of an 
exact image, without making most operation times increase. 
[0004] 

[Means for Solving the Problem] In order to attain the above-mentioned purpose, this invention moves a predetermined 
criteria image two-dimensional by the seek area in an input image, and it is each location of the above-mentioned seek 
area. In the image alignment approach which makes the location where the cross correlation function of the above- 



mentioned criteria image and the above-mentioned input image is calculated, and this cross correlation function serves 
as the greatest value the matching position of the above-mentioned criteria image and the above-mentioned input image 
* * * The above-mentioned cross correlation function is added about the lengthwise direction and longitudinal direction of 
the above-mentioned seek area, and it is constituted as the image alignment approach characterized by making into the 
above-mentioned matching position the location where the aggregate value of this lengthwise direction and the 
aggregate value of this longitudinal direction turn into the greatest value, respectively, furthermore, the above- 
mentioned aggregate value — the square of the above-mentioned cross correlation function ~ it is the image alignment 
approach which is the aggregate value of a value. 
[0005] 

[Embodiment of the Invention] It explains hereafter per gestalt of the operation which materialized this invention with 
reference to the accompanying drawing, and an understanding of this invention is presented. In addition, the gestalt of 
the following operations is an example which materialized this invention, and is not the thing of the character which 
limits the technical range of this invention. The block diagram showing the procedure of the image alignment approach 
0 which drawing 1 requires for the gestalt of operation of this invention here, and drawing 2 R> 2 are the graphs 
showing cross-correlation-function MATORIKUSSU concerning the image alignment approach 0. Moreover, SI, S2 
and S3 in drawing 1 , and S4 show procedure (step), and x and y express the longitudinal direction and lengthwise 
direction of an image, respectively. As shown in drawing 1 , the image alignment approach 0 concerning the gestalt of 
operation of this invention is the same as that of a Prior art at the point which is made to move a predetermined criteria 
image two-dimensional within the seek area in an input image, calculates the cross correlation function between the 
above-mentioned criteria image and the above-mentioned input image, and makes a decision criterion the greatest value 
in above-mentioned cross-correlation-function MATORIKUSSU. However, in the gestalt of operation of this 
invention, the above-mentioned cross correlation function is added about x directions and the direction of y (s2), and 
the aggregate value of x directions and the direction of y extracts the location which gives the greatest value, 
respectively (s3), and differs from a Prior art at the point made into a matching position (s4). Hereafter, the detail of the 
alignment approach 0 of the image concerning the gestalt of this operation is explained. First, in the seek area in the 
input image divided into the block of a MxN pixel, the criteria image divided into the block of a XxY pixel is moved 
two-dimensional. In this case, the coordinate at the upper left of a criteria image (a, b) moves in the range of (1 , 1) to 
(M-X +1, N-Y +1). Within the seek area, the cross-correlation coefficient C (a, b) in each location is calculated 
according to a degree type. 
[0006] 
[Equation 1] 
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[0007] In the above-mentioned formula, the value T of the input image whose coordinates the seek area of I is (x, y) in 
(a, b) on the strength That by which the values I and T of the criteria image whose coordinates are (x, y) on the strength 
were entitled - stores in Array R (a, b) the result of an operation of the cross correlation function from which the 
subscript of the average sigma of I and T was obtained by the formula of ******** of I and T (si). Next, Array R (a, b) 
is added in the x directions, SX (1) - SX (M-X +1) are added in the direction of y, and SY(1) -SY (N-Y +1) is obtained, 
respectively (s2). And in each, the location xO which takes the greatest value, i.e., a row number, and a line number yO 
are extracted among SX (1) - SX (M-X +1), and SY (1) - SY (N-Y +1) (s3). Finally, a row number xO and a line 
number yO are determined as a matching position (s4). Drawing 2 shows the line which takes the greatest value which 



can be set to the aggregate value of x directions by the result of an operation of procedure si and s2, and procedure s3, 
and each aggregate value of the direction of y , and the extract result of a train (a line and a train with the result of an 
4 * * operation shown with a thick frame). In addition, the point which is what is depended on a noise also has [ the result of 
an operation by procedure si ] the same cause that the location where the above-mentioned cross-correlation coefficient 
C (a, b) serves as max using the same result as drawing 4 used by explanation of a Prior art for a comparison is set to 
(xO-l,yO). 

[0008] In drawing 2 , as described above, it is not a location (xO-1, yO) but the location (xO, yO) which is determined 
that the image alignment approach 0 concerning the gestalt of this operation is a matching position. Since the image 
alignment approach 0 adds a cross correlation function about x and each direction of y and, as for this, both aggregate 
values make the matching position the location which serves as max at coincidence, It is because the judgment was 
seasoned with surrounding similarity, the location (xO-l, yO) [C(x0-1, yO) = 0.921] was dismissed and the location (xO, 
yO) [C(x0, yO) = 0.919] was judged to be the location which matched most synthetically. Thus, if this invention is used, 
it will become possible to perform alignment of an exact image which cannot be easily influenced by the noise. 
Moreover, the processing time which increased in the image alignment approach 0 is only the operation time in 
procedure s2 substantially. Since the operation in procedure s2 is very simple, it is extent which can be disregarded 
compared with the operation time in procedure si, and does not make most operation times increase. 
[0009] 

[Example] In the above-mentioned image alignment approach 0, although it dissociated, respectively and procedure si, 
s2, and s3 was performed, if the above-mentioned procedure si, s2, and s3 is mixed, the operation time can be 
shortened. Such an image alignment approach is also an example of the image alignment approach in this invention, 
moreover ~ although the cross correlation function was added as it was in the procedure s2 in the above-mentioned 
image alignment approach 0 - instead, the square of the above-mentioned cross correlation function ~ a value may be 
added. Such an image alignment approach is also an example of the image alignment approach in this invention. 
Moreover, in the above-mentioned image alignment approach 0, although the above-mentioned cross correlation 
function was calculated about all the coordinates in the above-mentioned seek area or [ limiting to the predetermined 
coordinate of the above-mentioned seek area by calculating about no coordinates from the beginning, when the above- 
mentioned seek area is large ] — or It flies at suitable spacing and evaluation by the above-mentioned image alignment 
approach 0 is performed about the field of a jump, as coordinate spacing is packed one by one, the operation time may 
be shortened, and alignment of an image may be performed. Such an image alignment approach is also an example of 
the image alignment approach in this invention. 
[0010] 

[Effect of the Invention] The image alignment approach concerning this invention can perform alignment of an image 
correctly, even when a noise exists in image information, without making the operation time almost increase compared 
with a Prior art since it is constituted as described above, further ~ the above-mentioned aggregate value - the square 
of the above-mentioned cross correlation function ~ the aggregate value of a value, then the difference in correlation 
are emphasized, and the alignment of a more exact image becomes possible. 
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TECHNICAL FIELD 



[Field of the Invention] This invention relates to the alignment approach of the image in the automation technique 
which used the image processing, for example, a character recognition technique, the automation technique of the 
visual inspection of a product, etc. 
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PRIOR ART 



[Description of the Prior Art] With the automation technique using an image processing, when it is the relation with 
which a predetermined criteria image (template) and a predetermined input image lap in a parallel displacement, the 
above-mentioned criteria image is moved within the seek area prepared into the input image, and the actuation which 
looks for the location which existed best, i.e., template matching, is used abundantly. In the case of matching, it is 
important to ask for whether the above-mentioned criteria image matches in which location of the above-mentioned 
input image correctly. For this reason, in the automation technique using an image processing, the cross-correlation 
coefficient method which calculates the cross correlation function between the above-mentioned criteria image and the 
above-mentioned input image has been used well, moving the above-mentioned criteria image within the seek area of 
an input image. Drawing 3 is drawing explaining template matching which used the conventional cross-correlation 
coefficient method. A cross-correlation coefficient C (a, b) calculates moving the seek area in the input image which 
consists the criteria image of a XxY pixel of a MxN pixel, as shown in drawing 3 (a). If the coordinate at the upper left 
of a criteria image (a, b) moves in the range of a coordinate (1 1) to a coordinate (M-X +1, N-Y+l) and the above- 
mentioned cross correlation function calculates, cross-correlation-function MATORIKUSSU as shown in drawing 3 (b) 
will be obtained, for example, in drawing 3 (b), since C (xO, yO) is max in 0.9, a criteria image and the same image 
exist in the location of a=x0 and b=y0 ~ ** (it matched) ~ it is judged. In addition, about such a Prior art, it is indicated 
by "image-processing handbook" p303-p304 of Morio Onoe Mr. edit, for example. 
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EFFECT OF THE INVENTION 



[Effect of the Invention] The image alignment approach concerning this invention can perform alignment of an image 
correctly, even when a noise exists in image information, without making the operation time almost increase compared 
with a Prior art since it is constituted as described above, further — the above-mentioned aggregate value — the square 
of the above-mentioned cross correlation function — the aggregate value of a value, then the difference in correlation 
are emphasized, and the alignment of a more exact image becomes possible. 
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TECHNICAL PROBLEM 



[Problem(s) to be Solved by the Invention] However, the conventional cross-correlation coefficient method which was 
described above is sensitive to a noise, and since the maximum of the above-mentioned cross correlation function 
becomes indistinct, there are many possibilities of obtaining the mistaken result. In addition, a noise here is a noise in 
the semantics of a wide sense including the difference produced since the lighting conditions when picturizing the 
difference which always exists between the others and above-mentioned criteria images and the above-mentioned input 
images which are generated with an image incorporation camera or an image processing system, for example, the 
above-mentioned criteria image and the above-mentioned input image, changed. [ noise ] The example from which 
matching exact for the above-mentioned noise is not obtained is explained using drawing 4 . Drawing 4 is a graph for 
explaining the incorrect judging in the conventional cross-correlation coefficient method with which the above- 
mentioned noise becomes a cause here. For example, in MATORIKUSSU of the cross correlation function shown in 
drawing 4 , although a cross correlation function takes the greatest value in a location (xO-1, yO), this is because the 
above-mentioned noise which existed in a criteria image and input image information lapped in the location [ by 
chance / (xO-1, yO) ], and right matching positions are (x0 5 yO). It sets in such a situation, and in the conventional cross- 
correlation coefficient method, since the decision of a matching position is made only with the greatest above- 
mentioned value and surrounding correlation is not taken into consideration, it will be judged that (xO-1, yO) are 
matching positions. In order to solve the technical problem in a Prior art which was described above, this invention 
improves a cross-correlation coefficient method, and it aims at offering the approach of performing alignment of an 
exact image, without making most operation times increase. 



[Translation done.] 



* NOTICES * 



/ - ' JPO and NCIPI are not responsible for any 

damages caused by the use of this translation. 

l.This document has been translated by computer. So the translation may not reflect the original precisely. 
2 **** s h ows wor d which can not be translated. 
3. In the drawings, any words are not translated. 



MEANS 



[Means for Solving the Problem] In order to attain the above-mentioned purpose, this invention moves a predetermined 
criteria image two-dimensional by the seek area in an input image, and it is each location of the above-mentioned seek 
area. In the image alignment approach which makes the location where the cross correlation function of the above- 
mentioned criteria image and the above-mentioned input image is calculated, and this cross correlation function serves 
as the greatest value the matching position of the above-mentioned criteria image and the above-mentioned input image 
The above-mentioned cross correlation function is added about the lengthwise direction and longitudinal direction of 
the above-mentioned seek area, and it is constituted as the image alignment approach characterized by making into the 
above-mentioned matching position the location where the aggregate value of this lengthwise direction and the 
aggregate value of this longitudinal direction turn into the greatest value, respectively, furthermore, the above- 
mentioned aggregate value ~ the square of the above-mentioned cross correlation function ~ it is the image alignment 
approach which is the aggregate value of a value. 
[0005] 

[Embodiment of the Invention] It explains hereafter per gestalt of the operation which materialized this invention with 
reference to the accompanying drawing, and an understanding of this invention is presented. In addition, the gestalt of 
the following operations is an example which materialized this invention, and is not the thing of the character which 
limits the technical range of this invention. The block diagram showing the procedure of the image alignment approach 
0 which drawing 1 requires for the gestalt of operation of this invention here, and drawing 2 R> 2 are the graphs 
showing cross -correlation-function MATORIKUSSU concerning the image alignment approach 0. Moreover, SI, S2 
and S3 in drawin gJL , and S4 show procedure (step), and x and y express the longitudinal direction and lengthwise 
direction of an image, respectively. As shown in drawing 1 , the image alignment approach 0 concerning the gestalt of 
operation of this invention is the same as that of a Prior art at the point which is made to move a predetermined criteria 
image two-dimensional within the seek area in an input image, calculates the cross correlation function between the 
above-mentioned criteria image and the above-mentioned input image, and makes a decision criterion the greatest value 
in above-mentioned cross-correlation-function MATORIKUSSU. However, in the gestalt of operation of this 
invention, the above-mentioned cross correlation function is added about x directions and the direction of y (s2), and 
the aggregate value of x directions and the direction of y extracts the location which gives the greatest value, 
respectively (s3), and differs from a Prior art at the point made into a matching position (s4). Hereafter, the detail of the 
alignment approach 0 of the image concerning the gestalt of this operation is explained. First, in the seek area in the 
input image divided into the block of a MxN pixel, the criteria image divided into the block of a XxY pixel is moved 
two-dimensional. In this case, the coordinate at the upper left of a criteria image (a, b) moves in the range of (1, 1) to 
(M-X +1, N-Y +1). Within the seek area, the cross-correlation coefficient C (a, b) in each location is calculated 
according to a degree type. 
[0006] 
[Equation 1] 
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[0007] In the above-mentioned formula, the value T of the input image whose coordinates the seek area of I is (x, y) in 
(a, b) on the strength That by which the values I and T of the criteria image whose coordinates are (x 5 y) on the strength, 
were entitled - stores in Array R (a, b) the result of an operation of the cross correlation function from which the 
subscript of the average sigma of I and T was obtained by the formula 0 f ******** of I and T (si). Next, Array R (a, b) 
is added in the x directions, SX (1) - SX (M-X +1) are added in the direction of y, and SY(1) -SY (N-Y +1) is obtained, 
respectively (s2). And in each, the location xO which takes the greatest value, i.e., a row number, and a line number yO 
are extracted among SX (1) - SX (M-X +1), and S Y (1) - SY (N-Y +1) (s3). Finally, a row number xO and a line 
number yO are determined as a matching position (s4). Drawing 2 shows the line which takes the greatest value which 
can be set to the aggregate value of x directions by the result of an operation of procedure si and s2, and procedure s3, 
and each aggregate value of the direction of y, and the extract result of a train (a line and a train with the result of an 
* operation shown with a thick frame). In addition, the point which is what is depended on a noise also has [ the result of 
an operation by procedure si ] the same cause that the location where the above-mentioned cross-correlation coefficient 
C (a, b) serves as max using the same result as drawing 4 used by explanation of a Prior art for a comparison is set to 
(x0-l,y0). 

[0008] In drawing 2 , as described above, it is not a location (x0-l, yO) but the location (xO, yO) which is determined 
that the image alignment approach 0 concerning the gestalt of this operation is a matching position. Since the image 
alignment approach 0 adds a cross correlation function about x and each direction of y and, as for this, both aggregate 
values make the matching position the location which serves as max at coincidence, It is because the judgment was 
seasoned with surrounding similarity, the location (x0-l, yO) [C(x0-1, yO) = 0.921] was dismissed and the location (xO, 
yO) [C(x0, yO) = 0.919] was judged to be the location which matched most synthetically. Thus, if this invention is used, 
it will become possible to perform alignment of an exact image which cannot be easily influenced by the noise. 
Moreover, the processing time which increased in the image alignment approach 0 is only the operation time in 
procedure s2 substantially. Since the operation in procedure s2 is very simple, it is extent which can be disregarded 
compared with the operation time in procedure si, and does not make most operation times increase. 
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EXAMPLE 



[Example] In the above-mentioned image alignment approach 0, although it dissociated, respectively and procedure si, 
s2, and s3 was performed, if the above-mentioned procedure si, s2, and s3 is mixed, the operation time can be 
shortened. Such an image alignment approach is also an example of the image alignment approach in this invention, 
moreover — although the cross correlation function was added as it was in the procedure s2 in the above-mentioned 
image alignment approach 0 — instead, the square of the above-mentioned cross correlation function ~ a value may be 
added. Such an image alignment approach is also an example of the image alignment approach in this invention. 
Moreover, in the above-mentioned image alignment approach 0, although the above-mentioned cross correlation 
function was calculated about all the coordinates in the above-mentioned seek area or [ limiting to the predetermined 
coordinate of the above-mentioned seek area by calculating about no coordinates from the beginning, when the above- 
mentioned seek area is large ] - or It flies at suitable spacing and evaluation by the above-mentioned image alignment 
approach 0 is performed about the field of a jump, as coordinate spacing is packed one by one, the operation time may 
be shortened, and alignment of an image may be performed. Such an image alignment approach is also an example of 
the image alignment approach in this invention. 
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DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing 1] The block diagram showing the procedure of the image<alignment approach 0 concerning the gestalt of 
operation of this invention. 

[Drawing 2] The table showing cross-correlation-function MATORIKUSSU concerning the image alignment approach 
0. 

[Drawing 3] The explanatory view explaining template matching in a Prior art. 

[Drawing 4] The table showing cross-correlation-fiinction MATORIKUSSU in a Prior art. 

[Explanation of agreement] 

x ~ Longitudinal direction in a seek area 

y ~ Lengthwise direction in a seek area 

a — X-coordinate at the upper left of a criteria image 

b — Y-coordinate at the upper left of a criteria image 

X — The pixel number of partitions of the x directions of a criteria image 

Y ~ The pixel number of partitions of the direction of y of a criteria image 

M ~ The pixel number of partitions of the x directions of an input image 

N — The pixel number of partitions of the direction of y of an input image 

si — Operation of a cross correlation function 

A cross correlation function is added about 2— sx direction and the direction of y. 
s3 — Maximum is extracted from an aggregate value. 
s4 ~ Matching position decision 
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